Discounted-Sum Automata with Multiple Discount Factors

Discounting the influence of future events is a key paradigm in economics and it is widely used in computer-science models, such as games, Markov decision processes (MDPs), reinforcement learning, and automata. While a single game or MDP may allow for several different discount factors, discounted-sum automata (NDAs) were only studied with respect to a single discount factor. For every integer ? ? ??{0,1}, as opposed to every ? ? ???, the class of NDAs with discount factor ? (?-NDAs) has good computational properties: it is closed under determinization and under the algebraic operations min, max, addition, and subtraction, and there are algorithms for its basic decision problems, such as automata equivalence and containment. We define and analyze discounted-sum automata in which each transition can have a different integral discount factor (integral NMDAs). We show that integral NMDAs with an arbitrary choice of discount factors are not closed under determinization and under algebraic operations. We then define and analyze a restricted class of integral NMDAs, which we call tidy NMDAs, in which the choice of discount factors depends on the prefix of the word read so far. Tidy NMDAs are as expressive as deterministic integral NMDAs with an arbitrary choice of discount factors, and some of their special cases are NMDAs in which the discount factor depends on the action (alphabet letter) or on the elapsed time. We show that for every function ? that defines the choice of discount factors, the class of ?-NMDAs enjoys all of the above good properties of integral NDAs, as well as the same complexities of the required decision problems. To this end, we also improve the previously known complexities of the decision problems of integral NDAs, and present tight bounds on the size blow-up involved in algebraic operations on them. All our results hold equally for automata on finite words and for automata on infinite words

On the Comparison of Discounted-Sum Automata with Multiple Discount Factors

We look into the problems of comparing nondeterministic discounted-sum automata on finite and infinite words. That is, the problems of checking for automata $A$ and $B$ whether or not it holds that for all words $w$, $A(w)=B(w), A(w) \leq B(w)$, or $A(w)<B(w)$. These problems are known to be decidable when both automata have the same single integral discount factor, while decidability is open in all other settings: when the single discount factor is a non-integral rational; when each automaton can have multiple discount factors; and even when each has a single integral discount factor, but the two are different. We show that it is undecidable to compare discounted-sum automata with multiple discount factors, even if all are integrals, while it is decidable to compare them if each has a single, possibly different, integral discount factor. To this end, we also provide algorithms to check for given nondeterministic automaton $N$ and deterministic automaton $D$, each with a single, possibly different, rational discount factor, whether or not $N(w) = D(w)$, $N(w) \geq D(w)$, or $N(w) > D(w)$ for all words $w$.Comment: This is the full version of a chapter with the same title that appears in the FoSSaCS 2023 conference proceeding

Discounted-Sum Automata with Multiple Discount Factors

Discounting the influence of future events is a key paradigm in economics and it is widely used in computer-science models, such as games, Markov decision processes (MDPs), reinforcement learning, and automata. While a single game or MDP may allow for several different discount factors, discounted-sum automata (NDAs) were only studied with respect to a single discount factor. For every integer $\lambda\in\mathbb{N}\setminus\{0,1\}$, as opposed to every $\lambda\in \mathbb{Q}\setminus\mathbb{N}$, the class of NDAs with discount factor $\lambda$ ($\lambda$-NDAs) has good computational properties: it is closed under determinization and under the algebraic operations min, max, addition, and subtraction, and there are algorithms for its basic decision problems, such as automata equivalence and containment. We define and analyze discounted-sum automata in which each transition can have a different integral discount factor (integral NMDAs). We show that integral NMDAs with an arbitrary choice of discount factors are not closed under determinization and under algebraic operations and that their containment problem is undecidable. We then define and analyze a restricted class of integral NMDAs, which we call tidy NMDAs, in which the choice of discount factors depends on the prefix of the word read so far. Some of their special cases are NMDAs that correlate discount factors to actions (alphabet letters) or to the elapsed time. We show that for every function $\theta$ that defines the choice of discount factors, the class of $\theta$-NMDAs enjoys all of the above good properties of integral NDAs, as well as the same complexity of the required decision problems. Tidy NMDAs are also as expressive as deterministic integral NMDAs with an arbitrary choice of discount factors. All of our results hold for both automata on finite words and automata on infinite words.Comment: arXiv admin note: text overlap with arXiv:2301.0408

Gonadotropic and Physiological Functions of Juvenile Hormone in Bumblebee (<i>Bombus terrestris</i>) Workers

<div><p>The evolution of advanced sociality in bees is associated with apparent modifications in juvenile hormone (JH) signaling. By contrast to most insects in which JH is a gonadotropin regulating female fertility, in the highly eusocial honey bee (<i>Apis mellifera</i>) JH has lost its gonadotrophic function in adult females, and instead regulates age-related division of labor among worker bees. In order to shed light on the evolution of JH signaling in bees we performed allatectomy and replacement therapies to manipulate JH levels in workers of the "primitively eusocial" bumblebee <i>Bombus terrestris</i>. Allatectomized worker bees showed remarkable reduction in ovarian development, egg laying, <i>Vitellogenin</i> and <i>Krüppel homolog 1</i> fat body transcript levels, hemolymph Vitellogenin protein abundance, wax secretion, and egg-cell construction. These effects were reverted, at least partially, by treating allatectomized bees with JH-III, the natural JH of bees. Allatectomy also affected the amount of ester component in Dufour's gland secretion, which is thought to convey a social signal relating to worker fertility. These findings provide a strong support for the hypothesis that in contrast to honey bees, JH is a gonadotropin in bumblebees and lend credence to the hypothesis that the evolution of advanced eusociality in honey bees was associated with major modifications in JH signaling.</p></div

The influence of JH on wax secretion.

<p><b>A and C.</b> The amount of wax secreted; <b>B and D</b>. the number of wax pots and cells constructed in a cage during the experiment. The replacement therapy included a single JH-III treatment in A and B, and two successive treatments in C and D. The p-values summarize the results of Kruskal-Wallis test; groups with different letters differ significantly in a Conover post-hoc test (p<0.05) test). Other details as in <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0100650#pone-0100650-g002" target="_blank">Fig. 2</a>.</p

The influence of allatectomy on ester amounts in the Dufour's gland secretion.

<p><b>A.</b> Total amount of secretion. <b>B</b>. The relative amounts of esters out of the Dufour's gland secretion. The Dufour's gland secretion was analyzed at the age of 7 days using gas chromatography/mass spectrometry (GC/MS). Shown are the mean ± SE, sample size within bars. The p-value summarizes the results of one-way ANOVA; groups with different letters differ significantly in a LSD post-hoc test (p<0.05).</p

The influence of JH on worker fertility.

<p><b>A.</b> The influence of JH on oocyte development. The values are median and 95% confident intervals for the length of the terminal oocyte. The replacement therapy included a single topical treatment with JH-III (CA-+JH¹). <b>C</b>. Same as A. but the replacement therapy included two successive treatments with JH-III (CA-+JH²). <b>B</b>. The influence of JH on egg-laying. The values are mean ± SE number of eggs laid in a cage accommodating a queenless group. <b>D</b>. Same as B. but the replacement therapy included two successive treatments with JH-III. Sample size is shown within or above bars and depicts the number of bees in A and B, and the number of cages (groups) in C and D. The p-value summarizes the results of Kruskal-Wallis test; groups with different letters differ significantly in a Conover post-hoc test (p<0.05). <b>E</b>. A photograph of representative ovaries from Experiment 3 (Summarized in panel C). For additional details see legend to <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0100650#pone-0100650-g001" target="_blank">Fig. 1</a>.</p

The influence of JH on fat body <i>Kr-h1</i> mRNA levels.

<p>For plot details see legend to <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0100650#pone-0100650-g003" target="_blank">Fig. 3A</a>.</p